AI, Python, Cognitive Neuroscience

BLEU might be Guilty but References are not Innocent - https://arxiv.org/abs/2004.06063 - We show that it is possible to calculate reliable automatic scores (even with BLEU) for high quality MT output by using a novel reference generation method.

Typical references exhibit poor diversity, concentrating around translationese language. Paraphrased references cover a wider diversity of target sentences and thus do not penalize alternative but equally accurate translations.

Releasing all reference translations gives the community a chance to revisit some of their decisions and measure quality differences for modeling techniques that produce more natural or fluent output which is penalized by standard references.

https://github.com/google/wmt19-paraphrased-references

🗣 @AI_Python_arXiv
✴️ @AI_Python_EN
❇️ @AI_Python

www.tg-me.com/ye/AI Python Cognitive Neuroscience/com.ai_python_en/2284

1.5K viewsFarzad 🦅, Apr 21, 2020 at 18:11

BLEU might be Guilty but References are not Innocent - https://arxiv.org/abs/2004.06063 - We show that it is possible to calculate reliable automatic scores (even with BLEU) for high quality MT output by using a novel reference generation method.

Typical references exhibit poor diversity, concentrating around translationese language. Paraphrased references cover a wider diversity of target sentences and thus do not penalize alternative but equally accurate translations.

Releasing all reference translations gives the community a chance to revisit some of their decisions and measure quality differences for modeling techniques that produce more natural or fluent output which is penalized by standard references.

https://github.com/google/wmt19-paraphrased-references

🗣 @AI_Python_arXiv
✴️ @AI_Python_EN
❇️ @AI_Python

BY AI, Python, Cognitive Neuroscience